Increasing Response Rates & Data Quality of Web Surveys: Pre-Notification and Questionnaire Paging Format

Although web-based survey research of the general population has serious limitations (e.g., non-coverage, lack of a sampling plan), it has been embraced by many who do survey research. It has been shown to be particularly useful when the population surveyed is almost universally connected to the Internet, and is proficient with the medium and the tools to use the medium (computing devices: e.g., desktop computers, PDAs, etc.). In this experiment conducted on a panel of electronics engineers and engineering managers, we explore issues that may affect response rates, and response quality. Respondents are asked to report on the process of designing an integrated circuit. First, we examine the effect of a time-honored method on both unit response rates and speed of delivery: a random half of the panel received a notification in advance of the survey invitation, informing them that they would be asked to participate in a few days; the other half did not. Second, we analyze what effect two different formats of the same instrument have on response quality. One half of the panel was randomly assigned to a scroll-type web questionnaire, in which the instrument is loaded onto one web page and the respondent scrolls down from one question to the next; the other half of the panel received a multiple page-type questionnaire, in which the respondents click on a button to move from one question to the next. Is one type more advantageous than the other? Measures of quality used will be the break-off rate (the proportion of respondents that abandon the survey) and item response rate (looking both at errors of omission, failing to answer a relevant question; and errors of commission, answering a question for which the respondent is not eligible).